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ABST^ \CT 


This thesis is a comparison of the capa cs currently available in the Joint 

Maritime Command Information System (JMCIS) to the data link requirements of the 
United States Marine Corps (USMC) Advanced Tactical Air Command Center (ATACC). 
The evolution of JMCIS and its underlying software design philosophy is discussed as 
well as the operational and financial advantages of this philosophy. The comparison of 
the AT ACC requirements and the JMCIS capabilities is done usinr simple 
Multi-Attribute Rating Technique (SMART). The SMART techniqj *:>signs weight 
values to the AT ACC requirements and calculates an overall comparison figure for 
JMCIS. The weight values were calculated fi^om survey data. Survey subjects pro\ided 
their perception to the relative mission criticality of the AT ACC requirements. The 
subjects for the evaluation were U.S. Marine Corps Officers with air command and control 
experience, and the evaluations were elicited using the Criterion DecisionPlus^ software 
package. The comparison figure for JMCIS averaged across the survey subjects was 
68%. The weighting fiictors and the model of the requirements revealed the shortfalls of 


the JMCIS system in the area of data link maintenance functionali^ 
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1. INTRODUCTION 


The system design philosophy behind the Joint Maritime Command Information 
system (JMCIS) is a revolutionary advancement in the de*/elopment of command and 
control systems. JMCIS provides the opportunity for significant improvements in 
operational capability, data interoperability, and human engineering with a substantial cost 
reduction. All these good things can come about through designing systems with the 
JMCIS philosophy and migrating current systems to this architecture. Yet it takes 
knowledge of JMCIS and the proposed migration system to bring these improvements to 
fruition. The information presented in this thesis can be used as a part of that knowledge 
to unlock the benefits of JMCIS. 

This thesis conducts a comparison between the capabilities currently available in the 
JMCIS system and the data link requirements of the Advanced Tactical Air Command 
Center (ATACC). The comparison method yields a numerical correlation figure 
representing the extent to which JMCIS meets the AT ACC requirements and identifies the 
marginal returns that would be gained by adding further functionality to JMCIS. 

A. SCOPE OF THESIS 

This thesis is a comparison of the capabilities currently available in the JMCIS to the 
data link requirements of the United States Marine Corps (USMC) Advanced Tactical Air 
Command Center (ATACC). The comparison is done using the Simple Multi-Attribute 
Rating Technique (SMART) as it is implemented in the software package Criterion 
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DecisionPlus™. The comparison of requirements to capabilities is weighted for relative 
importance of the requirements. This relative importance is derived from survey data 
collected from subjects that evaluated the importance of the requirements. The subjects 
for the importance evaluation were U. S. Marine Corps Officers with air command and 
control experience, and the evaluations were elicited using Criterion DecisionPlus™ 
software package. 

The origins of the JMCIS system and the Department of Defense policies that have 
shaped this software architecture are discussed to give the reader an appreciation for the 
development of JMCIS. Discussions of the benefits and current uses for the system are 
included in the thesis. 

B. THESIS ORGANIZATION 

1. Chapter II Introduction to ATACC 

In order to understand the structure of the comparison a knowledge of the 
Marine Corps Tactical Air Command Center's mission and organization is required. 
Chapter 11 defines the TACC's mission and gives the reader enough information about the 
staffing and functioning of the TACC in order to gain an appreciation for the use of the 
data link systems. The chapter explains the current configuration of the TACC with the 
AN/TYQ-1 equipment and also details the changes and improvements coming with the 
fielding of the Advanced Tactical Air Command Center (ATACC) with the AN/TYQ-51 
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equipment. For readers familiar with the TACC and the Marine Air Command and 
Control System (MACCS) this is review material. 

a. Appendix (A) Tactical Digital Information Links 

^pendix A is supplemental data of definitions and technical characteristics 
of the different types of Tactical Digital Information Links available to the TACC. This 
data provides further clarification to the Tactical Digital Information Links introduced in 
Chapter n. 

2. Chapter III JMCIS 

JMCIS provides the alternative data link capabilities that are evaluated in this 
thesis. Chapter m describes both the fielded JMCIS command and control system as well 
as the JMCIS philosophy. This chapter details the development of JMCIS and provides 
an explanation of the underlying software design philosophy for the readers unfamiliar 
with JMCIS. The evolution of the philosophy, and the command and control system, are 
traced through the developments and changes in Department of Defense policy. The 
lineage of the JMCIS system is traced back through the command and control systems 
fi'om which it evolved and a projection of the evolution of JMCIS in the future is given. 1 


1 Chapter ni is the product of a collaborative effort between researchers working on 
related JMCIS projects. Primary contributors include Lt. B. F. Loveless, USN., 
Lt. M. T. Weatherford, USN., and the author. 
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3. Chapter IV the AT ACC Requirements 


The first step in comparing the AT ACC data link requirements to the JfMCIS 
capabilities is to have a full understanding of the specified AT ACC requirements. The 
system requirements for the AT ACC were found in ELEX-T-620A dated 27 July 1990, 
and the contract modification to that document, P00068, dated 19 November 1992. This 
document became the source of the specific requirements that comprised the evaluation 
criteria for the JMCIS system. Chapter IV discusses the meaning of the specific 
requirements as well as the structuring of the requirements in the decision tree. The 
chapter identifies the meaning of the different requirement categories and the different 
levels within the decision tree. 

4. Chapter V the Comparison Method 

Chapter V provides an explanation for the selection of the Simple 
Multi-Attribute Rating Techiuque (SMART) as the method for rating the system and 
details how that technique is implemented in the software package Criterium 
DecisionPlus™. The required steps in using SMART are discussed as well as their 
manifestation in DecisionPlus^. These described steps illustrate to the reader the method 
used in building the decision tree as well as its use in capturing survey data fi'om the 
subjects. The chapter covers the organization of the decision tree, and the importance 
ranking procedures used to elicit data from the subjects. 
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a. Appendix (B) Single Multi-Attribute Rating Technique 
(SMART) 

Appendix B provides supplemental data for the background and the 
development of the SMART. This background information provides an understanding of 
SMART and illustrates why it was the appropriate technique for this comparison. 

b. Appendix (C) Criterium DecisionPlusF^ 

Appendix C provides details on how SMART is implemented in Criterium 
DecisionPlus^ and the operating characteristics of the program. This section also 
provides insight to the different user interfaces available in the software as well as other 
system capabilities. 

5. Chapter VI Alternative Evaluation and Comparison Results 

Chapter VI discusses the researcher's evaluation of the JMCIS system for 
implementation of low level functional requirements as well as the evaluation results. The 
chapter also clarifies calculations performed to arrive at a numerical score for the 
comparison of the JMCIS to the AT ACC requirements. The methods and the tools used 
to perform the analysis are discussed, as well as problems encountered using 
DecisionPlus™. 
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0 . Appendix (D) Supporting Data 

Appendix D is supporting numerical data that was used in the calculation of 
the comparison figures. The data includes the initial rating data, calculated intermediate 
steps, and other calculations. 

6. Chapter VII Conclusion 

Chapter VII summarizes the findings of the analysis of the data and reveals the 
areas where JMCIS did and did not meet the requirements. Related issues not covered in 
this thesis and other developing questions are discussed as potential research topics. 
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n. INTRODUCTION TO TACC AND ATACC 


The command center from where Marine Corps aviation assets are led and 
implemented is the Tactical Air Command Center (TACC). This chapter discusses the 
organization, mission, and equipment of the TACC. The capabilities of the current 
AN/TYQ-1 equipment is discussed as well as the improvements gained with the new 
AN/TYQ-51, or Advanced TACC (ATACC) equipment.2 

A. THE TACTICAL AIR COMMAND CENTER (TACC) 

1. Definition 

The TACC is the senior Marine Air Command and Control System (MACCS) 
agency. The TACC is the one MACCS agency which exercises command and it serves as 
the operational command post (CP) for the Aviation Combat Element (ACE) commander. 
The TACC provides the fricility from which the ACE commander and his battlestaff plan, 
supervise, coordinate and execute all current and future Marine Air Ground Taskforce 
(MAGTF) air operations. The TACC is operated and maintained by the ACE staff, 
personnel from the Marine Tactical Air Command Squadron (MTACS ), and the staff of 
the Marine Air Control Group (MACG). Liaison personnel from other Services may be 
required in the TACC for coordination of joint and combined operations. The Marine 

2 Major portions of this chapter are paraphrased from FMFM 5-60 (Control of 
Aircraft and Missiles), FNffM 5-5 (AntiAir Warfare) and selected Marine Corps 
Tactical Systems Support Activity (MCTSSA) information packages. 
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Corps Tactical Air Command Center (TACC) is sometimes called the Marine TACC to 
avoid confusion with the Navy Tactical Air Control Center (TACC). [Ref. 1 :p.3-l] 

2. TACC Organization 

The ACE commander directs and controls current and future operations from 
the TACC. Organic agencies of the MACG, support groups, and aircraft groups assist 
and implement the guidance of the TACC as well as non-organic agencies. Some of these 
agencies are; 


• The Tactical Air Operations Center (TAOC) from the Marine Air Control 
Squadron (MACS) 

• The Direct Air Support Center (DASC) from the Marine Air Support Squadron 
(M^S) 

• Marine Air TraflBc Control Squadron (MATCS) detachments 

• Stinger firing units from Low Altitude Air Defense (LAAD) Battalion 

• Hawk firing units from Light Anti-Aircraft Missile (LAAM) Battalion 

• Liaison ofiScers from other Services or nations. 

• Liaison officers from aircraft and support groups. 

• The Tactical Air Control Parties (TACP) organic to the Ground Combat Element 
(GCE). 

• Airborne controllers / coordinators, Airborne Supporting Arms Coordinator 
(SAC[A]), Airborne Tactical Air Coordinator (TAC[A]), Forward Air Controller 
Airborne (FAC[A]) [Ref l:p. 3-1] 


To facilitate this implementation of the ACE commander's direction and control 
of air operations the TACC is divided into two sections. Future Operations and Current 
Operations. 

c. Future Operations 

The term Future Operations refers to those activities directed against an 
enemy for which detailed planning must be accomplished and resources allocated. The 
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Future Operations Section (FOS) of the TACC accomplishes this detailed planning and 
allocation. Personnel in the FOS build the next Air Tasking Order (ATO) using 
preplanned requests and planing and coordination information coordinated with, and 
received from, the ACE HQ staff. The ATO is the document that apportions and allocates 
the MAGTF aviation assets to specific missions. Future Operations personnel focus on 
detailed planning and resource allocation for ACE support of the MAGTF for future deep, 
close and rear operations. [Ref 1 ;p. 3-2] 
b. Current Operations 

The term Current Operations refers to those activities directed against an 
enemy for which planning has been previously completed and resources committed. This 
is normally considered from the present time through the next 24 hours. These Current 
Operations include on-going operations such as deep, close and rear operations by the 
ACE in support of the MAGTF. Current Operations personnel execute the current Air 
Tasking Order (ATO). The ATO is a document that allocates the aviation resources to 
specific missions to be conducted. To accomplish this, the Current Operations Section 
(COS) communicates with the Future Operations Section (FOS) and other agencies to 
enable the direction and control of current operations. [Ref l:p. 3-1] 

3. TACC Tasks 

The role of the TACC is to function as the senior MAGTF air command and 
control agency, and to serve as the operational CP for the ACE commander. From the 
TACC, the battlestaff can supervise, direct, control, and coordinate the ACE's support of 
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the MAGTF's Current Operations and develop detailed plans for Future Operations. From 
the TACC, the ACE commander can plan and prosecute air operations to support the 
MAGTF commander's deep operations to isolate and prepare the battlefield. Also fi'om 
the TACC, the ACE commander can plan and prosecute air operations as the MAGTF's 
main effort or to support close and rear operations. [Ref l:p. 3-2] 

The tasks necessary to accomplish the role described above are many but can 
generally be described as maintaining situation awareness and providing tasking to 
subordinate agencies. While command is centralized for planning within the ACE HQ and 
the TACC, control is decentralized to subordinate MACCS agencies for specific aviation 
functions. Examples of this decentralization include the DASC's control of OAS 
(Offensive Air Support) and the TAOC's control of AAW (Anti Air Warfare) activities. 

4. Equipment Capabilities 

In order to accomplish the necessary tasks to fulfill the TACC’s roles, the 
Future Operations Section and the Current Operations Section require certain equipment. 
These equipment requirements can be categorized as either communication or display 
equipment. 

a. Communications 

(1) Voice 

The TACC has multiple voice communication circuits. A typical 
TACC configuration to support a Marine Expeditionary Force (MEF) might include (18) 
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ultra high frequency (UHF), (6) very high frequency (VHF), (18) high frequency (HF), 

and (20) multi-channel radio (MUX) circuits. 

(2) Data 

The TACC has the capability of communications over several Tactical 
I Information Link (TADIL) formats. These formats include TADIL-A, TADDL-B, 
anu jrth Atlantic Treaty Organization (NATO) Link-1. Joint Tactical Information 
Distribution System (JTIDS) or TADIL-J will be part of the system in the future. [Ref 


l:p. 3-5] 


A TADIL provides the means for the electronic transmission of 
specifically coded messages or comnumds from one agency to another and enables 
agencies to see information being provided by another's sensor. Tactical data exchange 
with other services is established on a mission or situation dictated basis. [Ref 1 :p. 10-5] 
Technical details and specifications of the different types of digital data 
links is contained in Appendix A. The TACC and the MACCS are normally connected 

with other services and agencies in the following manor; 

• TADIL-A with NATO and the Air Force Airborne Warning and Control Squadron 
(AWACS) or Tactical Air Control Squadron (TACS). 

• TADIL-A with the Navy ,Navy Tactical Data Systems / Airborne Tactical Data 
Systems (NTDS/ATDS). 

• TADIL-B with the Air Force (TACS). 

• TADIL-B with the Army , Army Air Defense Command Post (AADCP). 

• TADIL-C with appropriately equipped USMC/U.S. Navy (USN) aircraft (TAOC 
only). 

• NATO LINK-1 with NATO air control agencies. 

• ATDL-1 (Army Tactical Data Link) with Hawk units (TAOC only). 
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Figure 2-1 is an example of the typical data link connectivity emanating 
from a TACC. [Ref 1 :p. 10-6, Figure 10-2] With the capability to operate on different 
types of links and multiple data links at the same time, this figure represents only one 
possible connectivity diagram. The different types of links all have different strong points 
and weak points, thus units that can operate on a variety of links are more robustness and 
offer different options for connectivity or connectivity reconfiguration. 



Figure 2-1 Typical TACC Data Link Configuration 


b. Displays 

The TACC displays selected information necessary for coordination and 
supervision of MAGTF air activity. To provide this display capability the TACC uses 
manual status boards and electronic displays. [Ref l:p. 3-5] 
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Manual status boards are used to display data of a stable nature such as 
weather, communication status, aircraft availability, and ATO flight information. 

The electronic displays of the TACC have the capability to display selected 
air operations on a near-real-time basis in both graphical and tabular form. Data displayed 
includes air track information, weapon status, and map information.. Symbols representing 
aircraft, agencies, and geographic subdivisions are displayed to present a general picture of 
the air situation in the area of responsibility (AOR). These symbols or tracks are received 
from external radar surveillance agencies and command, control, communication, and 
intelligence (C3I) facilities for near-real-time information. [Ref 1 ;p 3-S] 

5. Relationships 

There is a coordinated relationship between the Navy TACC and the Marine 
TACC in order to conduct joint force operations. This relationship and the importance of 
information relayed via the Tactical Digital Information Links is described in FMFM 5-5, 
AntiAir Warfare as follows: 

The (Navy) tactical air control center is the primary air control agency for the 
Commander Amphibious Task Force (CATF) from which all AAW (AntiAir Warfare) 
means are controlled during the task force's movement to, and arrival at, the AOA 
(Amphibious Objective Area). Command relationships during the phasing of air 
control ashore AAW vary with the tactical situation. V^en the MACCS (Marine Air 
Command and Control System) is established ashore, a tactical digital information link 
(TADIL A)/Link 11 data link is established between MACCS AAW agencies and the 
tactical air control center afloat. Then, at a time mutually established by CATF and 
Commander Landing Force (CLF), control of AAW function is passed ashore The 
CLF exercises overall control through his tactical command center. At this time, the 
Tactical Air Control Center (afloat) reverts to a Tactical Air Direction Center and 
functions in a monitoring capacity ready to resume control if required. [Ref 2: CD 
version] 
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B. ADVANCED TACTICAL AIR COMMAND CENTER 
(ATACC) 

1. Definition 

The Advanced Tactical Air Command Central (ATACC)(AN/TYQ-51) is 
designed to replace the current Tactical Air Command Central Suite of equipment 
(AN/TYQ-1 and AN/TYQ-3A). The ATACC will provide a facility from which the 
Tactical Air Commander (TAC) and the Aviation Combat Element (ACE) battlestaff can 
supervise, coordinate and execute current and future tactical air operations over the 
Marine Air Ground Task Force's (MAGTF) airspace. Like the currently fielded 
AN/TYQ-1, the ATACC will be operated by the TAC, his staff, and designated personnel 
from the Marine Air Control Group (MACG). The ATACC is designed to support both 
the functions of the T ACC's Current Operations Section and the Future Operations 
Section. The personnel within the Current Operations Section focus on the current battle 
and deal particularly with a situation display, communications to other Marine and joint 
command and control agencies, and electronic status boards. The Future Operations 
Section is focused on planning for the future battle in 48-72 hours and produces the Air 
Tasking Order (ATO). These are the same functions done with the AN/TYQ-1 equipment 
however the ATACC was designed to provide the planner vdth automated planning tools 
and the ability to elearonically generate, disseminate and receive the Air Tasking Order 
(ATO). [Ref 3: p. 1] 
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2. Status 


The AT ACC provides significant operational and logistic enhancements over 
the AN/TYQ-1 equipment. It consists of two identical suites of equipment housed in 
shelters that measure 8 feet by 8 feet by 20 feet. Each suite is equipped with operator 
workstations, desktop communication units, a large screen display, radios, and other 
equipment necessary to perform aviation battle staff functions. This reduced logistical 
footprint enhances the capability to tactically reposition the equipment to meet changing 
missions and improve survivability. The importance of this maneuverability is echoed in 
FMFM 5-60, Control of Aircraft and Missiles, and in the Marine Corps Master Plan 
(MCMP) dated 21 July 1993. In these documents the requirement was identified for 
automated command and control (C2) systems with joint interoperability and connectivity 
to be of modular design and to be transportable by tactical vehicles. The most recent 
version of the Operational Requirements Document (ORD) specifies many of the desired 
improvements over the previous system. The improvements generally fall into the 
categories of logistical improvements, increased communication ability, and automation to 
support the generation a^d dissemination of the Air Tasking Order (ATO). The ORD 
document identifies phases of development where the AT ACC will evolve with increased 
capability over the different phases. [Ref 4:p. 1-34] 

Phase one of the ATACC is scheduled for delivery in 1996 and it will consist of 
a Grumman Data System module for the Current Operations Section and a suite of 
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CTAPS (Contingency Tactical Air Control System Automatic Planning System)3 
terminals for the Future Operations Section. Phase two of the AT ACC fielding plan is 
scheduled for the year 2000, and will involve fielding a system that integrates both of the 
functionalities into one console. [Ref 5] 


3 CTAPS is a United States Air Force command and control system that has 
become the default format for processing and disseminating Air Tasking Orders 
in joint operations. 
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ni. JOINT MARITIME COMMAND 
INFORMATION SYSTEM (JMCIS) 

To undo^tand the concept and the ptulosophy of JMCIS, the external evolutionary 
and developmental factors must first be examined. Changes in government and 
Department of Defense (DoD) information management policy and the complexion of the 
command and control systems absorbed under the JMCIS umbrella are the two defining 
elements in the evolution of JMCIS. 4 

A. POLICY 

The policies that have had the most significant impact in shaping the evolution of 
JMCIS are DoD's Corporate Information Management (CIM), The Joint Staffs "C4I for 
the Warrior", and the Navy's Copernicus architecture programs. These policies have 
contributed to the development of JMCIS by directing the evolution of the command and 
control environment fi'om which it evolved. 

1. DoD's Corporate Information Management (CIM) 

Defense Management Review Decision (DMRD) 918 provided the initial 
direction of the Corporate Information Management (CIM) initiative administered by the 
Defense Information Systems Agency (DISA). CIM is a strategic management initiative 
intended to guide the evolution of the DoD enterprise by capturing the benefits of the 
information revolution. It emphasizes both a functional and technical management focus 

4 Chapter HI is the product of a collaborative effort between researchers working on 
related JMCIS projects. Primary contributors include Lt. B. F. Loveless, USN., 
Lt. M. T. Weatherford, USN., and the author. 
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to achieve a combination of improved business processes and effective application of 
information technology across the functional areas of DoD. It is embodied in policies and 
programs, implement ^on guidance, and supporting resources, to help functional 
managers guide and implement changes to processes, data, and systems across the DoD. 
[Ref 6;p. 1] 

The management structure of CIM has four "pillars" that support improved 
Defense capabilities; common information systems; shared, standard data; re-engineered 
processes; and a computer and communications infrastructure. The overarching goal of 
CIM is to enable commanders of military forces and managers of support activities to 
achieve the highest degree of capability in their operations through the effective use of 
information applied in improved functional processes. The vision of this initiative provides 
for global end-to-end information cormectivity among U.S. and allied forces. In this 
context, information is considered a critical mission capability and force multiplier for 
worldwide readiness, mobility, responsiveness, and operations. Joint interoperability and 
information integration on the battlefield is emphasized to result in significantly improved 
joint service and multinational operations. [Ref 6;p. 3] 

2. The Joint Staff's "C4I for the Warrior" 

C4I for the Warrior is a concept for DoD information management first 
published by The Joint Staff in 1992. It is clearly targeted at solving the C4I 
interoperability issues among the services. The intent is to provide an unifying C4I 
concept that tvill support the requirements of the joint force Warrior at the battlefield 
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level, while remaining consistent with DoD policy and national security objectives. This 
focus is expressed by former Chairman, General Colin L. Powell, in the following 
statement; 

The C4I for the Warrior concept will ^ve the battlefield conunander access to all 
information needed to win in war and will provide the information when, where, and 
how the commander wants it. The C4I for the Warrior concept starts with the 
Warrior's requirements and provides a roadmap to reach the objective of a seamless, 
secure, interoperable global C4I network for the Warrior. [Ref 7-.p. 13] 

C4I for the Warrior is considered a seminal doctrine that is intended to guide 
the evolution of individual service C4I architectures into a broad Global Command and 
Control System (GCCS). [Ref 8;p. 49] The concept principles have been incorporated in 
the Joint StafPs GCCS program. 

At the center of the C4I for the Warrior concept is the establishment of a global 
C41 capability that allows the Warrior to define the battlespace and to "plug in" and "pull" 
timely, relevant information anytime, anyplace in the performance of any mission. The 
Warrior, by defining the battlespace, determines the information to "pull" rather than have 
information "pushed" fi’om various sources. The Warriors neither want nor need the 
cumulative knowledge of multiple sources dumped into their battlespace information 
systems. They want only the specific information they need to win the fight; and they 
want it when they need it, where thq' need it, and in the form in which it will do them the 
most good. This demand pull concept provides the capability for the Warrior to poll the 
global C4I network for any desired information fi'om any location, at any point in time. 
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This is a key principle of the C4I for the Warrior concept and a guiding concept for future 
DoD and Navy C4I architecture development. 

3. The Navy’s Copernicus Architecture 

The Copernicus Architecture is the current architectural guidance designed to 
restructure all Navy C4I systems. The Copernicus Architecture, Phase 1. Requirements 
Definition, published in 1991, provides both a new C4I architecture to replace the current 
Navy system and a programmatic investment strategy to construct it over the next decade. 
[Ref 9:p. 3-2] It is intended to establish a vision of an overall C4I architecture for the 
Navy. 

The Copernicus Architecture is primarily a telecommunications system designed 
around a series of glob./ information exchange systems ashore and tactical information 
exchange systems afloat. The architecture concept is based on four piUars. first, virtual 
global networks called Global Information Exchange Systems (GLOBIXS); second, 
metropolitan area networks called CINC Command Centers (CCC); third, tactical virtual 
nets called Tactical Data Information Exchange Systems (TADIXS); and fourth, 
interconnecting the previous systems to support the Tactical Command Center (TCC) 
afloat. In this concept, data can be forwarded fi'om the shore based sensor-to-sensor 
infi'astructure to the tactical commander's C2 infi^ructure afloat. Just as Copernicus 
brought about a revolutionary paradigm shift in astronomy, the Copernicus Architecture 
was so named because it represents a revolutionary paradigm shift in command and 
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control systems by being centered on the tactical needs of the operator afloat. [Ref 10:p 
10 - 12 ] 

A key operational concept of the Copernicus Architecture is the recognition of 
the Space and Electronic Warfare Commander (SEWC) as part of the Composite Warfare 
Commander (CWC) doctrine afloat. This action follows the establishment of SEW as a 
designated warfare area within the Navy by the CNO in 1989, which doctrinally assigned 
command and control (C2) functions to the SEW mission. In many ways, this early 
recognition of the importance of information management for the operational commander 
served as a building block for further DoD architecture development. The Copernicus 
goal of establishing a "common operating environment" now is considered part of the 
Defense Department's "C4I for the Warrior" initiative, which requires the Army, Navy, 
and Air Force to develop, through a phased process, approaches to making their C4I 
data-transfer systems fully compatible for joint operations. [Ref 8;p. 52] 

B. SYSTEMS 

JMCIS is an umbreUa system that has incorporated various functionalities and 
attributes of previous command and control systems. The philosophy of incorporating 
other systems capabilities and functionality is not unique to JMCIS, rather it is a trait 
inherited from previous systems. The Joint Operational Tactical System (JOTS), Navy 
Tactical Command System - Afloat (NTCS-A), and Operations Support System (OSS) are 
examples of systems that applied this same evolutionary methodology and directly 
influenced the development of JMCIS. 
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1. Joint Operational Tactical System (JOTS) 


JOTS began as a prototyping effort that was first deployed aboard ship in the 
early 1980s. This system provided the operational commander with the first integrated 
display of data for decision support purposes. System functionality eventually included 
track management, track analysis, environment prediction, and a variety of tactical 
overlays and Tactical Decision Aids (TDAs). JOTS was capable of receiving various data 
and message input such as Link 11, Link 14, Tactical Data Information Exchange 
System-A (TADIXS A), Officer in Tactical Command Information Exchange System 
(OTCDCS), High Interest Track (HIT) Broadcasts, and U.S. Message Text Format 
(USMTF) messages. JOTS allowed the Fleet Command Centers to interface with 
command ships and other shore instaUations. Through the use of a tactical data base 
manager (TDBM), JOTS provided a consistent tactical battlespace picture for all 
supporting warfare commanders afloat and ashore. [Ref 10;p. 60] 

The original prototyping effort of JOTS lead to the development of the JOTS 
Command and Control System by the late 1980s. The primary goal of the JOTS was to 
integrate information systems onto common hardware and software platforms to provide 
for the sharing of data bases as well as maximize limited shipboard area. JOTS-derived 
systems have since been installed onboard over 200 Navy ships, at several U.S. Navy 
shore intelligence centers, onboard U.S. Coast Guard vessels, onboard allied ships, and a 
various allied sites. [Ref ll;p. 1-1] As JOTS matured further and as other C31 systems 
were developed and deployed, it became apparent that there was much duplication of 
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software and ftinctionality across systems. This duplication led to increased development, 
maintenance, and training costs and the stated goal of interoperability across systems was 
virtually non-existent. This led to low interoperability and most importantly, led to 
conflicting information from multiple sources being provide to the operators. [Ref 11 p. 
1 - 1 ] 

2. Navy Tactical Command System - Afloat (NTCS-A) 

NTCS-A evolved from JOTS in the early 1990s, from the consolidation of a 
number of prototypes of individual "stovepipe" shipboard command and control software 
programs, including the Flag Data Display System (FDDS), the Joint Operations Tactical 
System (JOTS), the electronic Warfare Coordination Module (EWCM), and the Afloat 
Correlation System (ACS). [Ref 8;p. 52] Additional NTCS-A functionality was 
incorporated from other stand-alone or prototype C4I systems such as the Prototype 
Ocean SurveUlance Terminal (POST) and the Naval Intelligence Processing System 
(NIPS). Central to this consolidation effort was the abstraction of the afloat software into 
a common "core" set of software that could be used throughout the afloat community as 
the basis for their systems. This led to a set of common software originally called 
Government Off The Shelf (GOTS) version 1.1. 

The common core software concept was extended to the shore community to 
reduce development costs and ensure interoperability. This effort resulted in a collection 
of software commonly referred to as the Unified Build (UB) version 2.0 or GOTS 2.0. 
This software is now deployed both afloat, in NTCS-A and ashore, in Operations Support 
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System (OSS) or Navy Command and Control System-Ashore (NCCS-A). The strength 
of these two systems is that they are built on top of a common set of functions so that 
advancements and improvements in one area ^e immediately translatable to advancements 
in the other area. [Ref 11 :p. 1-1] 

3. Operations Support System (OSS) 

OSS is a system that evolved from the functionalities of the Navy World-Wide 
Military Command and Control System (WWMCCS) Standard Software, Operations 
Support Group Prototype, Fleet Command Center Battle Management Program, and 
JOTS. This system is considered the shore installation variant of NTCS-A and is often 
referred to as Navy Command and Control System-Ashore (NCCS-A). By migrating the 
OSS into the JMCIS architecture, the Navy is seeking management economies of scale 
and performance enhancements in OSS. 

C. JMCIS 

JMCIS represents the next logical step in the evolution of Navy C4I systems. The 
addition of functions to NTCS-A has led to the creation of a new version of that system, 
which has been designated the Joint Maritime Command Information System. [Ref 8:p. 
56] JMCIS is described as a "overarching architecture" that is still evolving as fleet 
operators refine C4I requirements and the functionality of other systems is migrated to the 
JMCIS architecture. The JMCIS approach to adding new functionality instead of building 
new systems allows the Navy to benefit from a single-configuration management 
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approach. The system software provides the basic function, such as display control, 
message tra£5c control, and specific applications for various classes of ship equipped. 

[Ref 8;p. S2] Programmatically, JMCIS has consolidated the functions of NTCS-A and 
its complimentary ashore program, the OSS. The two systems are expected to form a 
significant core of the ongoing development of DoD-wide C4I architectures, referred to as 
Global Command and Control System (GCCS), that will continue to consolidate the C4I 
initiatives of the individual services. [Ref 8;p. 52] 

1. Genesis and History 

JMCIS is the current state of C4I technology initially envisioned in 1981 by 
Vice Admiral (Ret.) Jerry O. Tuttle as the future of command and control. The JMCIS 
idea was cultivated fi’om efforts to evolve interoperable C3I systems that began in the mid 
1980's with the development of the Joint Operational Tactical System (JOTS) Command 
and Control System. The system was also designed to operate on the Tactical Advanced 
Computing (TAC) family of computers, as non-proprietary, open architecture that could 
be easily transported to subsequent improved versions of the TAC. [Ref 11 ;p. 1-3] 

Under the direction of SPAWAR (PD-60), the core software GOTS 1.1 was 
compiled for use throughout the afloat community as the basis for all C3I systems. GOTS 
2.0 was called the Unified Build (UB) 2.0 and was developed to include the ashore 
community to further increase C3I system interoperability. The Unified Build is 
confirmation of Vice Admiral (Ret.) Tuttle's recent statement. 
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The future of C4I ... wiU be built on a foundation of interoperability, open 
systents, and a common operating oivironment. 'Standardization* will be our battle cry. 
[Ref 12] 

2. System Migration 

On 1 November 1993, Assistant S«:retary of Defense (ASD) for C4I, Mr. 
Enunitt Paige, issued a memorandum requiring all DoD services to develop a detailed plan 
for migration of individual systems into a common C4I framework. All systems 
nominated for migration to a common framework were to be completed within three 
years. Those systems not designated by the respective service as a candidate for migration 
were to either cease to exist or apply for exception status. [Ref 13] Rear Admiral John 
Gauss of SPAWAR PD-60 stated that obsolete systems must be retired as soon as 
possible even if some functions have not been replaced due to the significant decreases in 
DoD funding. [Ref 14] The ASD memorandum brought the issue of a common C4I 
framework espoused in the C4I For the Warrior plan to the front. A form of this common 
C4I framework was in existence prior to the issuance of the memorandum and JMCIS is 
that architecture selected for the U S Navy and Marine Corps. Secretary Paige's 
memorandum accelerated existing Navy and Marine Corps migration planning and 
established JMCIS as a practical alternative for the other services. The legacy systems 
that were migrated into JOTS and eventually into JMCIS are depicted in Figure 3-1 [Ref 
15]. The systems that were initially migrated into JMCIS were operationally oriented and 
eventually this migration philosophy was extended to logistical and intelligence related 
systems. Table 3-1 provides a listing of the full names for the migrated systems. 
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3. WhatisJMCIS? 


JMCIS is a system built as an architectural framework to meet specific Navy 


and DoD command and control capabilities. Just like Microsoft Windows'™, JMCIS 


provides an environment for applications that consolidates common functions. In 


Windows™, multiple applications can share common utilities such as printing and file 


management, rather than duplicating those functions for each application. For command 


and control systems, JMCIS provides various common utilities including mapping, 


tactical database display, and cartographic functions among others. This collection of 


utilities comprises the JMCIS core and is graphictdly depicted as a part of the COE in 
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Table 3-1 MIGRATION SYSTEMS 


Abbreviation 

Full System Name 

NIPS 

NTCS-A Intelligence Processing Services 

JOTS 

Joint Operatimial Tactical System 

TFCC 

Tactical Flag Coninand Center 

ACS 

Afloat Correlation SyAem 

EWCM 

Electronic Warfare Coordmation Module 

POST 

Prototype Ocean Surveillance Terminal 

ATP 

Advanced Tracking Prototype 


Navy WMCCS Software Standardization 

FHLT 

Force High Level Sy^em 

OSS 

Operations Support System 

TSC 

Tactical Support Center 

STT 

Shore Targeting System 

CCSC 

Cryptologic Combat Support Console 

CCSS 

Cryptologic Combat Sui^xnt System 

cnvciu 

Cryptologic Interface Device/Unit 


Navy Tactical Command System • Afloat 

NAVSSI 

Navigation Sensor System Interface 

NITES 

NTCS-A Integrated Tactical Environmental Subsystem 

SSEE 

Ships Signal Exploitation Equipment 

SNAP 

Shipboard Non-tactical ADP Program 

MRMS 

Maintenance Resource Management System 

NALCOMIS 

Navy Aviation Ix^istics Command Management 
Information System 

NTCSS 

Navy Tactical Command Siqjport System 

BGPHES 

Battle Group Passive Horizon Extension System 

OBU/OED 

Ocean Surveillance Information System (OSIS) 

Baseline Upgrade 


Figure 3-2. [Ref. 11 ;p. 2-2] The core is maintained and expanded based upon the 
migration of legacy systems and improvements to existing JMCIS applications. The 
consolidation of common functions allows all applications to access the most efficient 
utility and provides the opportunity to easily update the core utilities with improved 
versions. In traditional client/server style, JMCIS servers provide core services to the rest 
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Figure 3-2 JMCISCOE 

of the LAN and each workstation may have both the same or different application 
software running. 

(L Components of JMCIS 

( 1 ) Applications 

Depicted vertically in Figure 3-2, ^plications access the JMCIS core 
services via Application Program Interfaces (APIs). In Figure 3-2 the applications 
annotated as 'Account Groups' are the standard applications that come as a part of JMCIS 
These house keeping applications are custom environments for the common activities of 
System Administration, Security Administration, Database Administration and the 
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standard JMCIS operator environment. The applications annotated as 'Segments' are a 
sample of some of the unique applications that have been developed or migrated into the 

JMCIS environment. The specific Segments listed represent; 

• SEWC - Space and Electronic Warfere Commander 

• STRIKE - Strike Plot 

• JOTS TDAS - Joint Operational Tactical System Tactical Decision Aids 

{ 2 ) Common Operating Environment (COE) 

The COE consists of the UNIX Operating System (OS), X Window 
graphical windowing system, and Motif standard styles, as well as core software for 
receiving and processing messages, correlation, updating the track database, and software 
for generating cartographic displays. [Ref 1 l:p. 2-1] 

( 3 ) Unified BuUd (UB) 

The UB is the foundation for all JMCIS software. The UB is a set of 
software components that include the Common Operating Environment (COE) and a 
standard software base for central applications and library functions necessary for basic 
command, control, and supporting functions. 

{ 4 ) Segment 

A segment is a software application that operates in the JMCIS runtime 
environment utilizing core functionalities for common operations. Segments access the 
core funaionality through a standard set of Application Program Interfaces (APIs). The 
standard set of APIs is managed by the core developers and is the access vehicle to core 
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functionality. Unique functionality for individual segments is provided by the individual 
applications source executable code. 

( 5 ) Variant 

A variant is a subset of segments, from the JMCIS Superset, installed 
for a specific mission area such as mission planning or battle group database management. 
The collection of various JMCIS segments are simply customized modules that define the 
JMCIS variant. 

b. The Three Perspectives of JMCIS 

( 1 ) Sailor / Soldier Perspective 

To the end user, JMCIS represents a Command Information System 
which is distributed across a Local Area Netwoilc (LAN) of workstations. Operators are 
able to access all required functionality from any workstation regardless of physical 
location or the actual location where the proces^g is taking place. The user is presented 
with only the functionality needed to meet their mission and other unneeded functionality 
is hidden to prevent overwhelming the user. An operator with a different set of tasks is 
presented with a different set of functionality but both operators perceive that the system 
looks and operates in the same way. JMCIS will appear to the operators as the identical 
Command Information System in use by military personnel in sister services with 
completely different mission objectives. This joint commonality is of increasing 
importance with the expanded role services are performing in the joint arena. [Ref 11 ;p. 
1-7] 
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Program Manager Perspective 

From the perspective of a military program manager, JMCIS presents 
the opportunity for an umbrella program which can encompass several programs. Faced 
with decreased funding, program managers can maintain program viability and achieve 
considerable savings by constructing their system from the JMCIS building blocks. In 
these times of budget austerity, this potential savings is sometimes the only feasible option 
for the programs. [Ref ll:p. 1-7] 

( 3 ) System Developer Perspective 

From the perspective of a system developer, JMCIS is an open 
architecture and a software development environment that offers a collection of services 
and already-built modules for Command Information Systems. The JMCIS developers 
provide detailed instructions on how to make applications or systems JMCIS compliant. 
These instructions include details on standard user inter&ce and the procedures for usmg 
core functionality via APIs. This core functionality has been previously developed and 
tested and therefore the developer need only produce components that are unique to their 
particular application. [Ref ll;p. 1-7] 

D. WHY JMCIS? 

The evolution to JMCIS was an operational and financial necessity in today's world 
of rapidly changing technology and decreased funding for DoD systems. JMCIS provides 
DoD with an opportunity to stay ahead of technological growth well into the next century 
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by implementing open systems architectures and ensuring standardization of software and 
hardware for C4I systems throughout the services. 

1. Operational Justification 

a. Command, Control, Communications, Conqjuters, and 
Intelligence (C4I) 

Command, control, communications and intelligence are pivotal to the 
success of any military mission. The addition of computers to the equation increases the 
fusion capabilities. The concept of computers being a force multiplier is espoused in the 
1993 C4I For The Warrior document. 

Fused information is more valuable to the Warrior than information received directly 
from separate, multiple sources to the degree that it provides the warrior with 'real 
truth.' [Ref 7;p. 13] 

More importantly, the ability to puU on demand, information from any location at any 
moment, gives the Warrior both more flexibility and the skill to tailor decisions to his 
specific needs. [Ref 7;p. 13] 

b. Technology Explosion 

Technological leaps are being experienced on an almost exponential scale. 
Rear Admiral Walter Davis, Head of the Warfrue Architecture and Systems Engineering 
Directorate at the Space and Naval Warfare Systems Command (SPAWAR) summed up 
the speed of the development of technology by saying that "...the commercial computer 
industry is introducing new systems and new capabilities approximately every 18 months." 
[Ref 8;p. 49-56] With the average DoD major automated information system (AIS) 
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acquisition taking over 24 months from requirements specification to system delivery, 

DoD is constantly being equipped with obsolete systems. Open systems architecture is 
the solution. The crux of open systems are common development standards from which 
products can be developed using nomproprietary specifications. The advantages of using 
open systems architect! an organization the size of DoD are profound and present the 

most efficient and practical approach to the use of hardware and software. 

One of the objectives of JMCIS is to avoid having command and control 
systems tied to a specific hardware platform or proprietary system. For this reason the 
JMCIS system is designed to operate on the family of TAC computers. The system is 
designed to be easily transported from one version of TAC computer to the next and be 
capable of exploiting the improved capability of the upgraded system. Rear Admiral 
Gauss stated that TAC hardware, COTS and GOTS software, and both government and 
industry standards, were to be used for all current and future JMCIS development. [Ref 
14] With the open architecture and commercial standards used by JMCIS, advances in 
computing platforms can be easily incorporated by simply changing the host machine for 
the system. Figure 3-3 presents the dramatic increase in the number of MIPS between 
successive TAC system procurements and the proposed processing capability of the 
TAC-4. [Ref 12, and 16] 

c. Shared Access to Common Data 

The Track Database is possibly the most important piece of the JMCIS 
Command Information System. This TDBM, coupled with the extensive communications 
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Common Engine Evolution 



DTC-l DTC-2 DTC-2 TAC-3 TAC-4 
(HP 920) (SUN 4/110) (SUN 4/300) (HP 750) (Projected) 


Figure 3-3 Platform Performance Improvements 
capabilities of JMCIS, fosters greater interoperability A\dth external sources and databases. 
The TDBM provides standard procedures and formats to add, delete, modify, and merge 
basic track data among the various workstations on the local area networks. With the 
increased capabilities of the TDBM to receive multiple sources of data, fusion of the 
information gives the warrior more intelligent correlation. [Ref 11 ;p. 2-20] 

2. Financial Justification 

Significant savings can be obtained by supporting a reduced number of lines of 
code. This reduction in lines of code is accomplished by implementing a conunon core of 
software and only producing the unique portions of the segment. Initial analysis of 
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candidate command and control systems eligible for migration to JMCIS revealed 
significant reductions in post deployment software support. 

o. Configuration Management - Hardware/Software 

The financial savings of moving toward an open architecture environment 
cannot afford to be overlooked. While hardware costs have experienced a steady 
downward trend over the last several years, costs for proprietary software have 
mushroomed. The use of COTS software products combats the problem of skyrocketing 
costs by allowing the developer of a product to spread the cost of development among all 
users of the product. Achieving these economies of scale is the major cost saving 
chai'acteristic of the JMCIS open architecture environment. Vice Admiral (Ret.) Tuttle 
noted that"... the expenditures on (software) applications • coding, debugging, and 
testing - spiral upwards to 90% of the total computer budgets." [Ref 12] 
b. Training 

In addition to the costs for hardware and software, the costs related to 
training are significant. Through the use of open architecture and standardization of 
human machine interfaces, both operator and maintenance personnel familiarization with 
one system will translate directly to other systems using TAC hardware and open 
architecture environments. The Common Operating Environment (COE) of JMCIS 
includes such standards as X Window and MOTIF style guide as well as the UNIX 
operating system. By training operators on these standard vendor products, the 
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fiuniliarization time for new personnel is limited to the minimum necessary to understand 
the new mission and results in more rapid improvement in operator performance. 


E. THE JMCIS PHILOSOPHY 

1. Don't Reinvent the Wheel 

If a component already exists, it should be utilized even if the component is not 
the optimum, best possible solution As early as 1987 a GAO report on the issue of 
interoper- ’ ’’ty among DoD C3I systems noted that; 

Solving this problem {of interoperability) is no easy task. . .. It will require a great deal 
of cooperation among the services and a genuine willingness on the part of each service 
to accept interoperability even when it conflicts with some traditional service practices. 
[Ref 17:p. 18] 

Almost any module can be improved but that is rarely the issue. For example, it 
is usually possible to obtain performance improvements in drawing speeds for cartographic 
displays by customizing designs to use hardware specific features. However, this may not 
be cost effective if platform portability is a requirement, or if performance gains are 
modest relative to perceived performance. [Ref 11 :p. 1-11] 

2. Existing Standards 

The commercial marketplace generally moves at a faster pace than the military 
marketplace and advancements are usually available at a faster rate. Use of commercial 
products has the advantage of lowering cost by using already built items, increases the 
probability of product enhancements because the marketplace is larger, and increases the 
probability of standardization. [Ref 11 ;p. 1 -12] 
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3. Interpretability 

Interpretation of standards are a major source of problems with interoperability. 
The way to combat the problem is to use identical software modules to perform common 
functions. This ensures that the same standards are applied to all users and therefore 
eliminates the opportunity for inaccurate or varying interpretations. [Ref ll:p.l-12] 

4. Focus Attention 

Focus efiPorts on the development of desired but currently unavailable 
functionality instead of re-generating existing capabilities. [Ref 11 ;p. 1 -12] 

F. THE OBJECTIVES OF JMCIS 

Given the philosophy and history of the JMCIS concept, there are a number of 
objectives which are immediately apparent. The objectives include technical 
considerations such as software reusability, enforcement of common "look and feel", and 
standardization of interfaces. These technical objectives in turn result in the potential for 
significant cost savings and development acceleration. 

1. Commonality 

Develop a common core of software that will form the foundation for Navy and 
Joint systems. 

2. Reusability 

Develop a common core of software that is highly reusable to leverage the 
investment already made in software development. 
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3. Standardization 


Reduce program development costs through objectives one and two and 
through adherence to industry standards. This includes the use of commercially available 
software components whenever possible. 

4. Engineering Base 

Through standardization and an open JMCIS architecture, establish a large base 
of trained software/systems engineers. 

5. Training 

Reduce operator training costs through enforcement of a uniform 
human-machine interface, commonality of training documentation, and a consistent "look 
and feel." 

6. Interoperability 

Solve the interoperability problem (at least partially) through common software 
and consistent system operation. 

7. Certification 

Provide a base of certified software so that systems performing identical 
functions will give identical answers. 
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8. Testing 

Increase the amount of common, reusable software to reduce testing costs 
because common software can be tested and validated once and then applied to many 
programs. [Ref ll;p.l-13] 

G. THE FUTURE 

The vision provided by strategic planning initiatives is being realized under the 
JMCIS banner. Systems continue to evolve toward the goal of an interoperable C4I 
system that focuses on support to the Warrior. The National Military Strategy Document 
(NMSD) for FY 1994-1999 establishes C4I as the overarching C4 programming objective 
and states that; 

Consistent with the C4I for the Warrior’ plar a’ mce and Agency programmed 
systems must be compatible and interoperable to support Joint and combined operation 
across the entire spectrum of conflict. [Ref 18] 

GCCS is a Joint Staff sponsored program envisioned by the C4I for the Warrior 
concept and represents the next step in the evolution of command and control systems. 
When fiilly implemented, GCCS will embody a network of systems providing the Warrior 
with a full complement of command and control capabilities. As part of the C4I for the 
Warrior concept, GCCS is evolving into the global, seamless "Infosphere" capable of 
meeting the Warrior's fused information requirements. [Ref 7;p.l3] 
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IV. THE ATACC REQUIREMENTS 


A. INTRODUCTION 

The first step in comparing the ATACC data link requirements to the JMCIS 
capabilities, is to have a full understanding of the specified ATACC requirements. The 
system requirements for the ATACC are found in ELEX-T-620A dated 27 July 1990, and 
the contract modification to that document, P00068, dated 19 November 1992. Only the 
data link requirements of the ATACC system were evaluated. The requirements for the 
ATACC were grouped into categories and formed into a decision tree with level zero of 
the tree being the goal of selecting a data link syston that meets the AT AC requirements. 
The requirements were first divided into the three categories of operational fimctions, 
maintenance functions, and performance standards. These three categories of 
requirements form level one of the decision tree, this section is depicted in Figure 4-1. 


Level 0 

Level 1 


Operational Functions 



Select Data Link Alternative 

^ - 

Maintenance Functions 



Performance Standards 



Figure 4-1 Decision Tree Goal Level and Level One 


41 
















This decision tree was used in determining the relative importance of each 
requirement and eventually used in the comparison of the JMCIS to the AT ACC data link 
requirements. The broad requirements categories were further broken down into level 
two categories and finally into level three categories. The level three requirements are the 
low level functional statements used in the evaluation. 


B. OPERATIONAL FUNCTIONS 

Operational Requirements are those requirements that specify some operational 
function be resident within the system or a particular function be performed in a specific 
manor. The overall analysis of the functional requirements yielded three level two 
categories of System Interface, Data Readout, and Data Link Capacity under the level 
one category of Operational Functions. The level two and three branches of the decision 
tree that fall under the category of Operational Functions is depicted in Figure 4 ~ 2 . 

1. System Interface 

Section 3.1.6.12.1, Software/Operator Interaction, of the AT ACC system 
specification gives the following general requirements; 

All software which interacts with an operator shaU utilize menus, icons, prompts, 
entry feed back, notices, windows, and summaries to guide the operator through the 
operation of the AT ACC. The use of the keyboard for other than text or data entry 
shall be kept to a minimum. The operator shall be provided a programmable function 
key capability. Menus, prompts, entry feedback, notices and summaries shall contain 
sufficient information in English or English abbreviations so that no requirement will 
exist for the use of hand-held lookup tables. [Ref 19;p. 62] 
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Figure 4-2 Level One Operational Functions 


Using this broad requirements statement and the amplifying remarks that 
followed the level two functional requirements of Prompts, Menus, and Display Aids 
were created under the level one category of system interfiu:e. 
a. Prompts 

Prompts shall be used when requesting the operator to enter variable data. 
Entry of valid data shall cause the display of menus, other prompts, entry feedback, or 
summaries. [Ref 19.p. 63] 
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b. Menus 


Menus shall be used to provide a collection of items form which an operator 
may make a single selection. The selection of any valid menu item shall cause the display 
of other menus, prompts, entry feedback, or summaries. [Ref 19:p. 62] 
c. Display Aids 

After system initialization the necessary display aids shall be provided to 
complete the entry of date and time, data link parameters, and data e?ctraction information. 
There snail be a provision for magnetic storage and recall of these entries. The data link 

parameters shall consist of the following; 

• Data Link Reference Point (DLRP) 

• Unit System Coordinate Center (USCC) 

• Unit Position (UPOS) 

• Unit Address (UADD) [Ref 19:p. 97] 

2. Data Readout (Hook Data) 

Section 3.1.6.2.2.1, Hook Data Readout, specifies that when a track is hooked 
by an operator at any workstation, information pertaining to the hooked track shall be 
presented in an area reserved on the face of the workstation. The system is required to 
display TADD^-A, TADIL-B, TADIL-J and NATO Link-1 tracks in a predetermined 
format. [Ref 19:p. 55] This level two requirement was broken down into only one level 
three functional requirement relating to forwarding of data link information in general. 
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3. Data Link Capability 

Section 3.1. S. 1.5, Digital Message Interface, specifies the required types of 
digital information links the system must be able to communicate on and the standards that 
must be obeyed. The level two requirement of Data Link Capabilities is broken down 
into three level three functional requirements. [Ref 19 ;p. 23] 

a. TADIL-J 

The AT ACC will be capable of operafmg on TADIL-J in accordance with 
IDH J’l'lDS TIDP-TE Vol. HI (Interface Desi^ Handbook, Joint Tactical Information 
Distribution System, Technical Interface Design Plan - Technical Edition, Volume m). 
[Ref 19 ;p. 23] 

b. NATO Link-1 

The AT ACC will be capable of operating on NATO Link-1 in accordance 
with Standardization Agreement or Standard NATO Agreement 5601 (STANAG). [Ref 
19 :p. 23] 

c. TADIL-B 

The AT ACC shall be capable of operating on TADIL-B in accordance with 
Joint Chiefs of Staff Publication 6-01.1(C) (JCS PUB 6-01.1(C)). [Ref 19 :p. 23] 

d. Link Forwarding 

All links will be capable of forwarding tracks from one link to another as 
specified in STANAG 5601, JCS PUB 6-01.1(C) and the Interface Design Handbook, 
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Joint Tactical Information Distribution System, Technical Interface Design Plan - 
Technical Edition, Volume m (IDH JTIDS TIDP-TE Vol. ffl.) [Ref 19 ;p 23] 

e. TADIL-A 

The AT ACC shall be capable of operatmg on TADIL-A in accordance with 
Joint Chiefs of Staff Publication 6-01.1(C) (JCS Pub 6-01.1(C)). [Ref 19 ;p. 23] 

C, MAINTENANCE REQUIREMENTS 

The level one requirements category of Maintenance Requirements consists of those 
items that are generally related to maintenance functions of the system or actions 
supporting some other operational function. The level one category of Maintenance 
Requirements was broken down into three level two categories of Data Extraction, Data 
Reduction, and Error Detection. The data extraction is analogous to taking a sample and 
the data reduction is analogous to analyzing that sample. That portion of the decision tree 
below Maintenance Requirements and down to the level three requirements is depicted in 
Figure 4-3. 

1. Data Extraction 

Section 3.1.6.12.7 of the ELEX-T-620A detuls the data management 
requirements of the system for data extraction. Data extraction is the process of taking 
samples of data flows or directing a copy of that data to some non-temporary storage 
medium for further analysis. The capability to extract data for further analysis is of little 

Figure 4-3 Level One Maintenance Functions 
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use, unless the operator has some control over selecting the extraction location, data type. 


and output devices. After analyzing the stated general requirements and the listed 
provisions for the level 2 requirement of Data Extraction, five level 3 functional 
requirements were determined. [Ref 19:p. 70] 

a. Annotation of Data 

The system is required to allow the operator to annotate the extracted data 
with a system time tag, extraction point indicator, link type designator, and channel 
number. [Ref 19 .p.70] 
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b. Starts Stop and Suspend 

The operator must have the ability to enter control information to start, 
stop, suspend, or terminate any particular extraction activity. [Ref 19;p. 70 ] 

c. Select by Output Device 

The operator must have the ability to define the output device, for example 
magnetic tape or magnetic disc. The operator must also be capable of defining the 
extraction file name. [Ref 19:p. 70] 

d. Select by Link Type 

The operator must be capable of defining the data type by link identifier. 
Examples of a link identifier are TADIL-A, TADIL-B, TADIL-J, and NATO Link-1. 
[Ref lO.p. 70] 

e Select by Point of Extraction 

The operator must have the capability to define the extraction point by link 
type and channel identifier and/or Central Processing Unit (CPU) channel identifier. The 
operator must also be able to select data as transmitted data or received data. [Ref 19;p. 
70] 

2. Data Reduction 

Section 3.1.6.12.8 of the ELEX-T-620A specifies the requirements of the 
system for data reduction. The reduction of extracted data is a maintenance tool used to 
determine the health of a data link, or a system, by analyzing a sample of the data. After 
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analyzing the stated general requirements for the level two category of Data Reduction, 
three level three functional requirements were determined. [Ref 19:p. 70] 

a. Specified Ou^ut Devices 

The operator must have the capability to designate the output device for the 
data reduction results. [Ref 19:p. 71] 

b. By File Name 

The operator must be capable of specifying by file name the source data to 
be analyzed. [Ref 19;p. 70] 

c. By Specified FUter Type 

The operator must be able to define the data to be reduced based upon filter 
entry. The selectable filters shaU be inclusive and additive and only data meeting the 
combined characteristics of the selected filters shall be reduced and output. These filters 
shall include link type, chaimel number and /or CPU channel identifier, time tag (fi'om 
Stan reduction, and to stop reduction), track number, message number, track identity, and 
identity amplifiers such as track type. [Ref 19:p.71] 

3. Error Detection 

Section 3.1.6.12.6 of ELEX-T-620A specifies that the system shall manage 
digital data communications to provide the capabilities necessary to suppon the exchange 
of digital data link information. This shall include the processing capability for message 
building, message interpretation, and error detection. [P ef 19;p. 69] Analyzing these 
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broad requirements and the accompanying conditions, the level two requirement of Error 
Detection was broken down into one level three functional requirement that was relevant 
to the data link requirements. 

a. Error Detection 

The system must provide the capabilities necessary to support the exchange 
of digital data link information, including error detection of messages for TADIL-A, 
TADIL-B, TADIL-J, and NATO Link-1. [Ref 19;p.69] 

D. PERFORMANCE REQUIREMENTS 

The level one category of Performance Requirements consists of those items in the 
system specification that dictate a specific level of performance or action. Relating to the 
data link requirements this section contains not just what types of links the system will 
communicate on but at what level of reliability, availability, maintainability and the data / 
track volume the system must maintain. The portion of the decision tree below 
Performance Requirements and down to the level three requirements is depicted in Figure 
4-4 

1. Maintainability 

Section 3.2.4 ofELEX-T-620A describes the maintainability requirements and 
delineates these requirements to the appropriate echelon of maintenance. These levels of 
maintenance are Organizational level (first and second echelon). Intermediate level 
(on-equipment, third echelon), and Intermediate level (off-equipment, fourth echelon). 
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Figure 4-4 Level Two Performance Standards 

The measures specified for each level of maintenance is the mean time to repair (MTTR) 
and the maximum corrective time (Met). [Ref 19;p.83] The level two requirements 
category of Maintainability was broken down into three level three functional 
requirements. 


























a. Mean Time To Repair (MTTR) First and Second Echelon 
(Organwitional Level) 

Organizational level maintenance (first and second echelon) shall be limited 
to maintenance tasks that do not require any special tools or test equipment. At this level 
preventive maintenance tasks including visual inspection, testing, cleaning and minor 
adjustments shall be done. The system shall be repaired by removal/replacement of faulty 
lowest replaceable units. A MTTR of no greater than 30 minutes and a Met of no greater 

than one hour at the 90th percentile shall be achieved. [Ref 19;p.83] 

b. Mean Time To Repair (MTTR) Third Echelon (Intermediate 
Level) 

At the intermediate level (on-equipment, Third echelon) maintenance shall 
be performed by diagnostics and by replacement / removal of faulty lowest replaceable 
units. These lowest replaceable units include black boxes and circuit card assemblies. A 
MTTR no greater than 30 minutes and a Met no grater than one hour at the 90th 
percentile shall be achieved. [Ref 19:p 83] 

c. Mean Time To Repair (MTTR) Fourth Echelon 

At the intermediate level (ofif-equipment. Fourth echelon) maintenance shall 
have the capability to repair selected lowest replaceable units. These lowest replaceable 
units include black boxes and circuit card assemblies. A MTTR no greater than one hour 
and a Met no greater than two hours at the 90th percentile shall be achieved. [Ref 19;p. 
83] 
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2. Reliability 


In section 3.2.3 of ELEX-T-620A, reliability is defined as the probability that 
the AT ACC shall complete its mission 24 hours a day for a minimum period of 30 days. 
The system specification prescribes a lower threshold of mean time between failure 
(MTBF) and the formula for calculating the reliability percentage. The level two 
requirements category of Reliability was broken down into two level three functional 
requirements. 

a. Mean Time Between Failure (MTBF) 

The system shall have a lower threshold of 348 hours MTBF, using the 
MIL-STD-781D definition of failures. [Ref 19:p. 82] 

b. Reliability Percentile 

The system shall operate for 24 hours a day for 30 days with an acceptable 
reliability percentage. The mathematical equation for calculating the reliability is; 

Where R = Reliability %, MTBF (lower) = 348 hours, m=720 hour 
mission, and "e'-Base of the natural logarithm. [Ref 19:p.83] 

3. Availability 

Section 3.2.5 of ELEX-T-620 defines availability as the probability that the 
AT ACC is totally operable at any random point in time. The level two requirements 
category of Availability was broken down to only one data link relevant functional 
requirement. [Ref 19;p. 84] 
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a. Availability Calculations 

The minimum inherent availability (Ai) of each suite shall be 0.999, based on 

specified reliability and maintainability requirements, expressed as a percentage ratio. The 

mathematical formula for the availability calculations is ; 

A -__ 0 999 

~ MTBFiMTm ^ 

Where the MTBF is the Mean Time Between Failure and MTTR is the 
Mean Time To Repair. [Ref 19 :p. 84] 

4. Data Through-put 

Section 3.2.1.9.3 of ELEX-T-620A specifies the channel bit rates required of 
the system for the different digital information links. This level two requirements category 
is broken down into four level three functional requirements corresponding to the different 
links. [Ref 19;p. 82] 

a. TADIL-A 

The system shall implement TADIL-A and maintain a channel data rate of 
2,250 bits per second (bps) half duplex and a message rate of 1800 bps. [Ref 19 .p. 82] 

b. TADIL-B 

The system shall implement TADIL-B and maintain a channel data rate of 


1,200 bps full duplex and a message rate of 800 bps in and 800 bps out. [Ref 19;p. 82] 
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c. NATOLink-l 

The system shall implement NATO Link-l and maintain a channel data rate 
of 1,200 bps full duplex and a message rate of920 bps in and 920 bps out. [Ref 19;p 
82] 

d TADIL-J 

The system shall implement TADIL-J and maintain a channel data rate of 
28,800 - 23,800 bps half duplex and a variable message rate of 1,219 bps (min.) in/out and 
2,211 (max.) in/out. [Ref. 19:p. 82] 

5. Data Link Track Capacity 

Section 3.2.1.1 of ELEX-T-620A describes the minimum track capacity 
required of the system. This level two requiremoits category is broken down into five 
level three functional requirements. 

a. JTAO Tracks 

The system must process data representing a minimum of 500 JTAO and 
NATO tracks. [Ref 19;p. 74] 

b. Ground Tracks 

The system must process data representing a minimum of400 ground 
tracks. [Ref 19:p. 74] 

c. Engagements 

The system must display at least 100 engagements and at least 100 pairings. 
[Ref 19;p. 74] 
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d. Fixed Marks 


The system must display at least 40 fixed and at least SO internal 
communication marks, and 50 external pointers. [Ref. 19;p. 74] 
e. Track Growth Capacity 

The system must have the growth capacity to grow fi'om 500 JTAO and 
NATO tracks up to 1000 tracks. AdditionaUy the ground tracks must have a growth 
potential to go fi’om 400 up to 600 tracks. [Ref 19:p. 74] 

6. Multiple Data Link Capability 

Section 3.2.1.9.2 ofELEX-T-620A, specifies the numbers of simultaneous data 
links that the system must accommodated. The level two requirements category of 
Multiple Data Link Capability is broken down into only one, data link relevant, level three 
functional requirement. 

a. Multiple TADIL-B Links 

The system must be capable of processing nine TADIL-B links 
simultaneously. [Ref 19;p. 81] 
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V. THE COMPARISON 


There are several academically accepted methods for performing a comparison of the 
data link requirements for the AT ACC to the capabilities found in JMCIS. Some of these 
methods are; the Analytical Hierarchy Process (AHP), Multi-Attribute Utility Theory 
(MAUT), and the Simple Multi-Attribute Ratting Technique (SMART). For this 
comparison SMART was chosen based upon its simple and straight forward calculations 
and elicitation methods. The comparison of the requirements was done using a weighting 
factor for the AT ACC requirements based upon their importance to operators. Having the 
ability to accept weighted assignments was another reasons why SMAP.T was the favored 
choice. 

Using the Simple Multi-Attribute Rating Technique (SMART) and its 
implementation in the software package Criterium DecisionPlus^, a model of the decision 
was made The model was used to make a comparison between the A TACC requirements 
and the JMCIS capabilities. In order to use Criterium DecisionPlus™, software the task 
had to be reduced to a decision between at least two alternatives based upon multiple 
attributes. In this instance the multiple attributes were the AT ACC requirements, and the 
alternatives were the JMCIS System and an ideal system. This ideal system was assumed 
to be a system that meets all of the AT ACC lequirements at the stated level and nothing 
more. The ideal system will obviously meet the AT ACC requirements and got the 
maximum score from the model because it was built precisely to meet the requirements. 
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However, the distance between the score for JMCIS and the score for the ideal systein 
will give an indication of how closely the JMCIS capabilities meet the AT ACC's data link 
requirements. 

A. SIMPLE MULTI-ATTRIBUTE RATING TECHNIQUE 
(SMART) 

The Simple Multi-Attribute Rating Technique (SMART) was developed by Dr. 

Ward Edwards in 1977. It can be considered a derivative of the Multi-Attribute Utility 
Theory (MAUT) of which versions can be traced back as far as 1959. SMART is 
simplified in that it uses easier more straight forward measurement and elicitation 
techniques than MAUT. SMART ignores measurement theory and nonadditives and 
instead relies on simple additive models, numerical estimation techniques for eliciting 
single-attribute values and ratio estimation of weights. There are several different versions 
of SMART but ail have in common the reliance upon direct numerical estimation methods. 
[Ref 20:p. 278] 

Appendix (B) provides a more detailed discussion of the development and 
details of SMART, including the list of the ten steps associated with SMART 

B. CRITERIUM DECISIONPLUS^“ SOFTWARE 

Criterium DecisionPlus™ is a Microsoft Windows™ based program designed to be 
an analysis tool to aid in complex decision making tasks. This software is designed to 
support individual decisions, group decisions, and research findings. The software 
implements both the Analytical Hierarchy Process (AHP) and the Simple Multi-Attribute 
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Rating Technique (SMART) as selectable rating techniques. The user fnendly mouse 
driven environment provides simplified elicitation of subjects rating opinions, performs 
numerical aggregation, weighting calculations, and generates selectable reports and 
graphs. 

The software supports a brainstorming feature where the user can enter a goal, and 
alternatives to achieve that goal, on a blank canvas. The user then can connect the goal to 
attributes relevant to that goal and relationships are established. The finished brainstorm 
session can be used to automatically generate a value tree or hierarchy tree which 
represents the decision scenario. 

DecisionPlus™ provides a criterion rating environment where the user is given one 
of several selectable rating views to enter their evaluation to assign weights to the 
attributes entered in the brainstorming session. The weighted criterion are aggregated and 
used in determining the desired alternative. The data from the evaluation is finally used in 
several reports, graphs and tables. A more detailed discussion on the capabilities and the 
steps for using DecisionPlus™ is contained in Appendix (C). 

C. ANALYSIS METHODOLOGY 

Using the DecisionPlus™ software a decision scenario was constructed using the 
brainstorming feature. During the brainstorming process four steps need to be completed. 

These four steps are : 

• Define a goal. 

• Define alternatives. 

• Identify relevant criteria. 

• Establish the relationships between criteria, subcriteria and the goal. 
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These four steps were the key decisions in designing the scenario in the 
brainstorming function. The researcher defined the goal and alternatives in order to meet 
the research objectives. The relevant criteria were selected fi-om the AT ACC system 
specification based upon their relevance to data link operations. The relationships 
between the criteria was established by the researcher according to functionality and the 
detail of the criteria. Completing the four steps, the brainstorming se^. then used 

to automatically generate a decision hierarchy. 

1. Defining a Goal 

Using the brainstorming feature of DecisionPlus™ the first step was to establish 
a goal for the decision. The goal for this decision scenario was to choose an alternative 
data link system for the Marine Corps ATACC. 

2. Define Alteraatives 

With the goal of the decision scenario established, the alternatives to meet that 

goal must be defined. The alternatives for this decision scenario were defined as; 

• A JMCIS system with its included data link capabilities 

• An ideal system that was assumed to have met all of the requirements specified in 

the ATACC system specification. 

3. Identify Relevant Criteria 

The relevant criteria relating to the decision goal of selecting an alternative data 
link system for the Marine Corps ATACC were the data link related requirements from the 
ATACC system specification. These data link related requirements are detailed in Chapter 
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4. Establish Relationships Between Criteria, Subcriteria and Goal 

To establish relationships between the criteria and the goal, the criteria were 
grouped into major functional categories and separated into three levels. The decision tree 
generated with the different levels, alternatives and the goal is depicted in Figure 5-1. 

5. Evaluating the Importance of Categories and Criteria 

Having established the goal, alternatives, criteria, and relationships the decision 
model was completed. At this point the model depicts relationships but the relationships 
are not evaluated. Referring again to Figure 5-1, when evaluating the level one and level 
two criteria the evaluation is on categories of functional capabilities rather than the 
capabilities themselves. In evaluating these two levels the subjects evaluate one criteria at 
a time and score the relative importance of that criteria against the other criteria at that 
level. When evaluating the level three functional criteria, subjects repeat the process and 
rank each criteria for its relative importance among the other level three criteria. After 
evaluating the relative importance, DecisionPlus™ facilitates the evaluation of each of the 
level three criteria for their level of implementation in the alternatives. More succinctly 
put, all criteria and categories are scored for how important they are compared to others 
at their level, and then the alternatives are scored on how well they implement the level 
three criteria. 
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Figure 5-1 Decision Tree 


62 



























































The evaluation of the relative importance of the three levels of criteria was 
conducted using the Criterion Rating environment in DecisionPlus’^ The subjects for the 
evaluation were Marine Corps Of5cers with recent Marine Air Command and Control 
experience. All of the subjects had been assigned to a Marine Air Control Group and have 
had experience with digital information links in the Marine Corps.4 The subjects only 
rated the relative importance of the level one, two, and three criteria and did not rate the 
alternatives for the level three criteria. The alternatives were scored by the researcher 
following an in-depth study of the JMCIS system. 
a. Evaluation View 

DecisionPlus™ provides the options of presenting the subject with three 

different views of the Criterion Rating environment. The researcher has the choice 

♦ 

between a graphical view, numerical view, verbal view, or a combination of the three The 
graphical view presents a sliding bar to the user that can move by mouse input. The 
numerical view presents the user with an entry window to enter a number and it informs 
the user of the acceptable range of numbers. The verbal view presents the subject with 
five rating level categories. DecisionPlus™ provides six, default groups of categories for 
the researcher to choose fi'om, or a custom scale can be created. The view used to 
evaluate the importance of the AT ACC criteria was the verbal view with a scale of 
Critical, Very Important, Important, Unimportant, and Trivial. The verbal view was 

4 All Marine Officers within the Marine Air Control Group with the military 
occupational specialty of 7202, 7204, 7208, 7210, and 7323 arc eligible for 
assignment to the Marine TACC and arc familiar with data link operations. All 
subjects came from the 72XX communities. 
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selected based upon several reasons. In addition to the categories the verbal view 
provides a descriptive sentence that seems to serve as a continuous reinforcement to the 
user as to the purpose and the context of the current evaluation. An example of the 
evaluation window used is provided in Figure C-2 of Appendix C. 

The five categories of the verbal view are more limited than the possible 
inputs firom the graphical view or the numerical view, however based upon the findings of 
Elmore & Beggs (1975), the increase fi-om 5 to 7 or 9 points on a Likert type scale does 
not statistically improve the reliability of the ratings. [Ref 21 :p. 134] Therefore the 
increased numbers of possible inputs was sacrificed in order to facilitate easier solicitation 
of responses fi'om the subjects. 

6. Evaluation of the Alternatives 

The decision hierarchy generated by the brainstorming session was presented to 
the subjects for the evaluation of the importance of the categories and criteria. The 
evaluation of the functional criteria for the alternatives was already completed by the 
researcher. The ideal system (or perfect system) had been given a maximum score for 
implementing all level three criteria. The JMCIS system was scored by the researcher 
based upon evaluations done in coordination with the JMCIS developers at Naval 
Research and Development (NRAD) and hands on experience. This section of the model 
was pre-scored based upon the subjects not having been exposed to JMCIS and not 
having a full understanding of its capabilities. This also added consistency to the 
interpretation of the functional requirements and the JMCIS capabilities. 
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VI. ALTERNATIVE EVALUATION AND 
COMPARISON RESULTS 

This cluster discusses the evaluation of the level three requirements in the JMCIS 
system as well as the logic used to determine the scoring. The steps used in processing 
the survey data and the calculation methods used to reach the JMCIS correlation figure 
are presented. 

A. SCORING THE JMCIS SYSTEM 

The capabilities of the JMCIS system were evaluated and compared to the level 
three functional requirements. The level three requirements were individually evaluated 
and scored as a "yes" or a "no" in the DecisionPlus™ software. Yes, the system has a 
capability that meets the stated requirement, or. No the system does not have a capability 
that meets the stated requirement. The methods used for determining the scores ranged 
from literature reviews, interviews with system developers, and hands on experience. In 
instances where the JMCIS capabilities were defined by different methods than the 
standards specified in the AT ACC requirements document, attempts were made to 
normalize the comparison. In cases where the comparison could not be normalized the 
researcher's judgment was the deciding factor. 

B. SCORING OPERATIONAL FUNCTIONS 

Under the level one category of Operational Functions there were three level two 
functional categories. These level two categories were System Interface, Data Readout, 
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and Data Link Capability. Table 6-1 is a summary of the Score of the Operational 
Functions. 


Table 6-1 OPERATIONAL FUNCTIONS SCORE 


System Interface 

Prompts 

Yet 


Menus 

Yes 


Display aids 

Yes 

Readout 

Hook Data 

Yes 

Link cq>ability 

TADIL-J 

Yes 


NATO Link 1 

No 


TADE^B 

No 


Link Forwarding 

No 


TADIL-A 

Yes 


1. System Interface 

The functional capabilities grouped under System Interface were, Menus, 
Prompts, and Display Aids. These items generally describe a set of user friendly operator 
to machine interaction conventions. The JMCIS ^stem was designed to conform with 
version 3.0 of the DoD Human Computer Interface Style Guide. The specific 
implementation of this style guide in the JMCIS system is specified in the User Interface 
Specifications For the Joint Maritime Command Information System version 1.3, 
November 1993. [Ref 22;p. 1-4] After reviewing this document and considering hands 
on evaluation of a stand alone system, the JMCIS system was evaluated as "yes” to all the 
fiinrtional requirements under System Interface. 


66 




2. Data Readout (Hook Data) 

The system specification for Data Readout relates to the display of track data 
fi-om the different data links in the specified format. The JMCIS system displays data from 
multiple sources to include some data links. Accordingly, the JMCIS system was scored 
"yes" for the requirements under Data Readout. 

3. Data Link Capability 

The system specifications grouped under Data Link Capability list the specific 
types cf data links the system must be capable of performing. As discussed in Chapter m, 
the origins of the JMCIS system show that it had its beginnings with the U.S.. Navy 
shipboard community. For this reason the system incorporates TADIL-A and the newly 
developed TADIL-J. Additionally, since the JMCIS predecessor JOTS was run in parallel 
with the older NTDS systems (Naval Tactical Data Systems) the systems were only used 
in a receive mode and did not transmit track information. 

For TADBL-A the JMCIS system is capable of receiving and displaying data 
from a link terminating device. There are three devices fielded today in the Navy. The 
Passive Link Tap (PLT), the Link Eleven Display System (LEDS) and the EDO box 
produced by EDO of Chesapeake, Wginia. [Ref 23] These three link terminating 
devices provide the JMCIS system with a one way, or receive only capability for 
TADIL-A. An upgrade to the JMCIS system has been developed and is being fielded in 
the Navy's Tactical Support Centers (TSC) to give the system a two way, receive and 
transmit, capability on TADIL-A. [Ref 23] The link terminating device for TADIL-J is 
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the Joint Tactical Information Distribution System (JTIDS) terminal. Currently the Navy's 
Advanced Combat Direction System (ACDS) ships equipped with block zero software 
have the capability for one way, or receive only TADIL-J. Ships equipped with ACDS 
and block one software have the capability for two way or, receive and transmit, capability 
on TADIL-J. [Ref 23] 

Accordingly the JMCIS system was scored "yes" for TADIL-A and TADIL-J , 
and scored a "no" for NATO Link-1, TADIL-B, and Data Link Forwarding. 

C. SCORING MAINTENANCE REQUIREMENTS 

Under the categoiy of Maintenance Requirements the three level two categories 
were Data Extraction, Data Reduction, and Error Detection. Table 6-2 is a summary of 
the score of the Maintenance Requirements. 


Table 6-2 MAINTENANCE FACTIONS SCORE 


Data Reduction 

Specified output devices 

No 


By Gleruune 

No 


filter entry 

No 

Data Extraction 

Armotation of data 

No 


Start, Suspend, terminate 

No 


Select by output device 

No 


Select by link type 



Select by extraction pt 


Error Detection 

building, interpretation and error 
detection of me 

Yes 


1. Data Extraction 

The requirements under Data Extraction in the AT ACC specifications generally 
deal with the capability to extract a sample of data for future analysis. The specific 
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requirements in this section deal with capabilities regarding the control of taking that 
sample data and the storage, marking and maintaining that data. 

The JMCIS system was not designed with a data extraction capability 
specifically intended for data link communications. The JMCIS system was designed to 
communicate and share data over a variety of links and communication paths. The system 
does have the capability to view incoming data and route that data fi'om an incoming port 
to another out going port. It is conceivable that a form of data extraction could be done 
by routing an incoming data stream to an external port and capturing that data with some 
other recording device. [Ref 23] A data extraction of this method would not provide for 
the specified control and annotation capability detailed in the AT ACC requirements. 
Accordingly the JMCIS system was scored a "no" for all of the functional requirements 
under data extraction. 

2. Data Reduction 

The data reduction capability is normally considered the processing of the data 
collected or sampled during the data extraction process. The JMCIS system was scored 
as "no" for all of the requirements under Data Reduction since the system has neither the 
capability to take samples nor analyze them. [Ref 23] 

3. Error Detection 

The funrtion of error detection for data links is not contained in the JMCIS 
system. However, considering the combination of the JMCIS system and the appropriate 
link terminating equipment there is con.siderable error checking . For TADIL-A 'he error 
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detection is done in either the PLT, ELDS, or EDO Box and for TADIL^J the error 
detection is done at the JTIDS terminal. [Ref 23] Therefore the JMCIS system was 
scored as a "yes" for the requirements under Error Detection. 

D. SCORING THE PERFORMANCE REQUIREMENTS 

Under the level one category of Performance Requirements there were six level two 
requirements categories of Maintainability, Reliability, Availability, Data Through-put, 
Data Link Track Capacity, and Multiple Data Link Capability. Table 6-3 is a summary of 
the Performance Requirements Score. 

1. Maintainability 

The AT ACC system specification describes the maintainability requirements and 
delineates these requirements for the appropriate echelon of maintenance. These levels of 
maintenance are Organizational level (first and second echelon), Intermediate level 
(on-equipment, third echelon), and Intermediate level (off-equipment, fourth echelon). 

The JMCIS system does not delineate maintainability by echelon of maintenance but rather 
by MTTR for hardware and MTTR for software. The JMCIS criteria for these 
MTTR is < 1.00 hour for hardware and < 20 minutes for software. [Ref 16:p. 12] These 
times can be roughly considered equivalent to the AT ACC requirements and therefore the 
JMCIS system was scored "yes" for the maintainability requirements. 
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Table 6-3 PERFORMANCE STANDARDS SCORE 


Maintainabilit)' 

MTTR OOmin 3id echelon 

Yes 


MTTR <lbr 4tb echeloD 

Yes 


MTTR <ygani7ational 

Yes 

Reliability 

24fan X 30days 

Yes 


348hrMTBF 

Yes 

Availability 

Ai=.999 

Yes 

Thnxi^i^mt 

2250bps TADIL-A 

No 


1200bps TADIL-B 

No 


1200bps NATO-1 

Yes 


28.8-23.8kbpsTADIL-J 

Yes 

Track CqMcity 

500 JTAO tracks 

Yes 


400 Ground Tracks 

No 


100 Engagements 

No 


40 Fixed marks 

No 


Track cap. growdi 

No 

Multi Links 

9 TADIL-B Links 

Yes 


2. Reliability 

The AT ACC system specification for reliability details a lower threshold of 
mean time between fiulure (MTBF) of348 hours, and the formula for calculating the 
reliability percentage. The JMCIS system criterion specifies a separate MTBF for 
hardware (> 800 hours) and MTBF software (>200 hours). [Ref 24;p. 11] After 
evaluating the differences between the two system requirements the JMCIS system was 
scored "yes" for the reliability requirements. 

3. Availability 

The ATACC system specification defines availability as the probability that the 
AT ACC is totally operable at any random point in time. The minimum inherent 
availability (Ai) of each AT ACC suite shall be 0.999, based on specified reliability and 
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maintainability requirements, expressed as a percentage ratio. The criterion availability for 
the JMCIS system is ^ .96. In an operational evaluation of NTCS-A version 2.0, the 
version that preceded JMCIS, the demonstrated operational availability was 0.89 aboard 
USS KITTY HAWK and 0.99 aboard USS COWPENS. [Ref 24:p. 12] After 
considering the differences in the availability rates and the different calculation methods, 
the JMCIS was scored as a "yes" for the requirements under Availability. 

4. Data Through-put 

The system requirements grouped under Data Through-put specify the speed at 
which the different data links must pass data. The JMCIS system was scored "yes" for 
TADIL-A and TADIL-J and for all others was scored "no". [Ref 23] 

5. Data Link Track Capacity 

The requirements grouped under Data Link Track Capacity generally deal with 
the minimum numbers of the different types of tracks the system must be able to display. 
The different categories of tracks are: JTAO Tracks, Ground Tracks, Engagements, and 
Fixed Marks. The specifications also list the desired Track Growth Capacity. The JMCIS 
system is capable of displaying 2000 OTH Gold tracks and any combination of 500 data 
link tracks. Considering the system capability the JMCIS system was scored a "yes" for 
JTAO Tracks, and Ground Tracks, and was scored as "no" for Engagements, Fixed Marks 
and Track Capacity Growth. [Ref 23] 



6. Multiple Data Link Capability 

The functional category of Multiple Data Link Capacity refers to the section of 
the system specification where the specific numbers of data links the system must be 
capable of performing at the same time. The requirement specifies that the system be 
capable of operating on nine different TADIL-B links at the same time. Recognizing that 
the JMCIS system caiuiot operate on any TADBL-B data links, the system was scored as 
"no" for this requirement. [Ref 23] 

E. SURVEY RESULTS 

The elicitation methods described in Chapter V were used to gain data firom the 
survey subjects. U.S. Marine Corps Officers with previous command and control 
experience comprised the survey sample. The subjects all previously had spent time 
working in a Marine Tactical Air Command Center (TACC) or Tactical Air Operations 
Center (TAOC), and were familiar with the Tactical Digital Information LinL* used by the 
Marine Corps. The survey elicited opinion data firom ^ subjects. The results derived 
from a sample of this size were not intended to be statistically significant, rather they are 
intended to illustrate the comparison methodology rather than the results. 

The software package DecisionPlus^ gathered the individual rating factors from the 
subjects and also calculated the overall weighting functions for the scoring of the 
alternatives. The software provided a list of weights by criteria and an overall score for 
both the JMCIS System and the Ideal System for each subject. 
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1. Score Calculation Process 

The scores were calculated by DecisionPlus™ in a method that weighed the 
presence of a hinctional criteria based upon the subjects impression of the criteria's 
importance. 

a. Ratings to Weighed Criteria 

DecisionPlus’^ recorded the subjects rating of each of the level one, two 
and three criteria. The ratings for the individual criteria were converted to the level three 
weighted criteria by multiplying the level three rating by the parent level two ratting and 
the level one parent ratting. The resulting set of level three weights all sum to one. This 
normalized list of weights was considered as the weighted importance of the level three 
hmaional requirements. 

b. Alternative Scoring 

The scoring of the JMCIS system and the Ideal system was also done in 
DecisionPlus™. This scoring was conducted by the researcher and the scale was a 
dichotomous yes or no decision. The yes or no score indicated whether the alternative 
system could, or could not meet the specified requirement. This scoring on the 
dichotomous scale yielded a ratting value of zero for a no response, and one for a yes 
response. The requirement scores as a group represent the by requirement evaluation of 
the alternative systems. 
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c. Individual Overall Score of the System 

The score of the alternative systems on each criteria was determined by 
multiplying the weighted importance of the level three flmctional requirements by the 
appropriate requirement score. This operation yielded the score of the system for that 
criteria and the sum of all the criteria is the overall score of the system. The ideal system 
was scored as yes on all of the criteria and therefore the sum of the criteria scores was 
one. The overall score of the system was calculated by DecisionPlus™ for the individual 
sets of data. 

d Average Ratings Set 

DecisionPlus^ has the capability to link several individual rating models 
into an aggregated result. This method of linking was attempted and a calculation error in 
DecisionPlus^ was detected. [Ref 25] The logic of the data aggregation model was 
recreated in a Lotus 123^ spread sheet and the individual rating data was exported from 
DecisionPlus™. The individual responses to each rating were averaged to come up with 
an average set of ratings for the group. 

e. Average Weighted Importance of Level three Requirements 

The average ratings were multiplied in the same manor as the individual 
ratings (Level three rating *Parent Level two ratting *Parent Level one ratting) to come 
up with the average weighted importance of the level three functional requirements. 
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/ Average Overall Score of the System 

The average overall score of the system was calculated in the same method 
as the individual score of the system with the exception of using the average weighted 
importance of level three requirements vice the individual weights. The data and the steps 
used while generating the average overall system score for JMCIS is provided in Table 

6-4. The table consists of four columns of data labeled and calculated as follows; 

• JMCIS Score: represents the researchers dichotomous evaluation of JMCIS for 
the level three requirements. 

• Avg Rating: represents the Average Rating which is the average of each of the 
subjects rating value given for that requirement. 

• Std. Dev: represents the Standard Deviation of the rating values for a specific 
requirement. 

• Avg. Weight: represents the average weighting factor for that requirement. It is 
calculated by multiplying the average level three rating by its parent level two and 
one average rating value. 

Appendix D provides a complete listing of the individual and average data. 

F. ANALYSIS OF DATA 

Table 6-4 depicted the average ratings of the criteria, the score of the level three 
criteria, and the overall score of the JMCIS system. There are a total of 34 level three 
functional requirements. Of these 34 functional requirements the JMCIS was evaluated as 
meeting 17 and not meeting 17. The 17 requirements that JMCIS did fulfill accounted for 
a score of .67 out of a possible perfect score of 1.00. Let us now turn our attention to not 
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Table 6-4 AVERAGE OVERALL SYSTEM SCORE CALCULATIONS 
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what the system does but what it does not do. The 17 requirements that were not fulfilled 
are distributed among the level one functional categories as follows; 

• three (3) fi^om Operational Functions 

• eight (8) fi-om Maintenance Functions 

• six (6) fi’om Performance Standards 

Rather than look at the unfulfilled requirements as they relate to the level one 
functional categories, a more meaningful measure is to group the requirements by 
similarities from within the group of 17. Categorizing the requirements based upon 
similarities the 17 unfulfilled requirements can be assembled into seven groups. Table 6-5 
depicts the consolidation of these requirements into the seven groups with the individual 
contribution and the group total contribution. The groups are listed in the order of highest 
group total to lowest group total. Rather than dealing with the 17 unfulfilled requirements 
individually, this table depicts the major categorical shortcomings of the JMCIS system. 
Additionally it depicts where the largest improvement in score could be gained when 
deciding to add new functionality to JMCIS. 

The seven groups of unfulfilled requirements are; 

• Data Extraction Group 

• Data Reduction Group 

• Multiple Links 

• Forwarding 

• NATO Link Group 
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Table 6-5 RANKING OF MISSING FUNCTIONALITY 


Data ExtractioB Group 

Annotkoon <iatt 

Siait, Su^, Suqieiid 

Select by output device 

Select by link type 

Select by extraction pt 

Data Reduction Group 

Specified output devices 

By filename 
filter entry 

Multiple links 

9TADIL-BLinks 

Track Capacity Group 

100 Engagements 

40 Fixed marks 

Track cap. growth 

NATO Link Group 

NATO Link 1 

1200bps NATO-1 

TADIL-B Group 

TADIL-B 

1200bps TADn-B 

Forwarding 

An. Total Bv Group % 

Wcfarfit Group of Totftl 




0.0156 



0.0] S4 



0.0156 



0.0173 



0.017 

0.083834 

27.612 







0.0287 



0.0215 



0.0261 

0.076317 

25.136 







0.0478 

0.047778 

15.736 







0.0079 



0.0089 



0.0123 

0.02902 

9.5581 







0.0137 



0.0144 

0.028074 

9.2465 







0.01 



0.0149 

0.024975 

8026 







Link Fotwaiding 

0.0136 

0.013617 

4.485 





Total Points for Unfulfilled 




Requirements f 


0J03616 
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• TADIL-B Group 

• TADIL-B 

• 1200 bps TADIL-B 

• Track Capacity 

The grouping of the unfulfilled requirements in this manor illuminates the fact that 
the major shortcomings of the JMCIS system came under the level two category of 
maintenance functions. The missing maintenance functions alone account for over 50% of 
the missing points. If the system were to implement the maintenance functions of data 
extraction, data reductiort, and tlie required control features, the overall system score 
would go fi'om 0.68 to 0.85. 
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VII. CONCLUSIONS 


A. THE FINDINGS 

By usi ig the Simple Multi-Attribute Rating Technique to conduct a comparison of 
the capabilities found in JMCIS with the AT ACC data link requirements, a numerical 
score was calculated. This figure represents the percentage of functionality required by 
the ATACC specifications that is found in the JMCIS system. The score is weighted to 
represent a higher percentage value for the requirements evaluated as more mission critical 
by a survQT of subject area experts. Combining the authors evaluation of the JMCIS 
functionality and interpretation of the ATACC specifications with the subject experts 
evaluations, the comparison method revealed a 68% correlation. 

The requirements that were evaluated as not being met by the JMCIS system 
compromise the remaining 32%. Closer evaluation of these unfulfilled requirements 
reveals that over half of them are maintenance related requirements in the areas of Data 
Extraction and Data Reduction capabilities. 

B. FURTHER RESEARCH 

This comparison has attempted to measure the commonalty between a set of 
requirements and the capabilities within JMCIS. The methodology used in this 
comparison represents an alternative method for assessing the potential systems to be 
migrated to the JMCIS environment. The evolutionary process of command and control 
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systems migrating to the JMCIS environment normally begins with an analysis of the 
required functionality. This functionalit)’ analysis in the past has been focused on what 
functionality will reside m the common core, and what system unique functionality will be 
maintained in an application segment to JMCIS. The modeling approach taken in this 
thesis could be used on a larger scale to determine trends in the un&lfilled requirements 
across several systems. The scores from candidate systems could be compared by 
conducting an analysis similar to this thesis before and after functions common to the 
systems were added to the core. This would represent the value of adding those functions 
to the core. 

The author presents the JMCIS philosophy toward ^stem engineering which 
revealed several key questions that routinely arise during system migration. Currently, 
there is much work underway involving system migration and analysis of what systems 
would make good migration candidates. These questions and the search for better ways 
to answer them will be at the forefront of system engineering for some time to come. The 
benefits achieved by the system design philosophy that gave birth to JMCIS are key to the 
elusive improvements sought on numerous fronts. For this reason, any other research 
efforts that attempt to provide better or alternative methods for comparing systems or 
system functionality will be of benefit to the community. 
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APPENDIX (A): TACTICAL DIGITAL 
INFORMATION LINKS 

The definitions of the different types of data links as listed in Joint Publication 1 
(DOD Dictionary of Military and Associated Terms, JP 1-02) and in FMFM 3-30 
Communications, 3 April 1989, arc provided as follows: 

A. TADIL 

A Tactical Digital Information Link is a Joint Staff approved, standardized 
communication link suitable for transmission of digital information. The current practice 
is to characterize a tactical digital information link (TADIL) by its standardized message 
formats and transmission characteristics. TADILs interface two or more command and 
control or weapons systems via a single or multiple network architecture. Multiple 
communication media can be used for the exchange of this tactical information. [Ref. 
26:CD version] 

B. TADIL-A 

TADIL-A is a secure, half-duplex, netted digital data link utilizing parallel 
transmission frame characteristics and standard message formats at either 1364 or 2250 
bits per second. It is normally operated in a roll-call mode under control of a net control 
station to exchange digi,il information among airborne, land-based, and shipboard 
systems. Data from sensors such as radar is processed, then time multiplexed on either 
HF or UHF for transmission to all participants in the net. TADIL-A utilizes the M-series 
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message standard described in JCS Pub 6-01.1(C) and its NATO equivalent is Link 11. 
[Ref. 26:CD version] 

C. TADIL-B 

TADIL-B is a secure, full-duplex, point-to-point digital data link utilizing serial 
transmission frame characteristics and standard message formats at either 2400, 1200, or 
600 bits per second. It interconnects tactical air defense and air control units. TADIL-B 
utilizes the M-series messages standard described in JCS Pub 6-01.1 (C). [Ref. 26:CD 
version] 

D. TADIL-J 

TADIL-J is a secure, high capacity, jam-resistant, node-less data link which uses 
the Joint Tactical Information Distribution System (JTIDS) transmission characteristics. 
The JTEDS protocols, conventions, and fixed-length message formats defined by the 
JTIDS Technical Interface Design Plan (TIDP) are also used. The spread spectrum 
(Frequency Hopping) system uses the JTIDS (Hass 2 Time Division Multiple Access 
(TDMA) terminal to broadcast J-scries messages to all / specific participants. [Ref. 

26: CD version] 

E. NATO LINK 1 

NATO Link 1 (North Atlantic Treaty Organization Link 1) or NADGE Link 1 
(NATO Air Defense in the Ground Environment Link 1) is a NATO point-to-point 
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digital data link. This link utilizes serial transmission frame characteristics and standard 
message formats at a speed of 600,750,1200, or 1500 bits per second. [Ref. 27p. 44] 
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APPENDIX (B): SIMPLE MULTI-ATTRIBUTE 
RATING TECHNIQUE 

The Simple Multi-attribute Rating Technique (SMART) can be considered a 
derivative of the Multi-attribute Utility Theory (MAUT) of which versions can be traced 
back as far as 1959. In 1971 Dr. Ward Edwards knew of the theory behind MAUT but 
was frustrated with its complicated measurement and elicitation techniques it seemed to 
require. Dr. Edwards thought that some set of simple and robust procedures would be 
better than the theoretical soundness and elegance of MAUT. His answer was SMART. 
SMART ignores measurement theory and non-additives and instead relies on simple 
additive models, numerical estimation techniques for eliciting single-attribute values and 
ratio estimation of weights. There are now several different versions of SMART but all 
have in common the reliance upon direct numerical estimation methods. [Ref. 20:p. 278] 
In Dr. Edwards article "How to Use Multi-attribute Utility Measurement for Social 
Decisionmaking", IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-7, 

No 5, May 1977, the following ten steps to SMART were identified: 

1. Identify the person or organization whose utilities are to maximized 

2. Identify the issue or issues to which the utilities needed are relevant. 

3. Identify the entities to be evaluated. 

4. Identify the relevant dimensions of value for evaluation of the entities. 

5. Rank the dimensions in order of importance. 

6. Make ratio estimates of the relative importance of each attribute relative to the 
one ranked lowest in importance. 

7. Sum the importance weights; divide each by the sum. 

8. Measure the relative value of each entity (alternative, object) on each dimension 
on a scale ofO to 100. 

9. Calculate the overall values using a weighted additive model. 

10. Choose the alternative that maximizes the overall value. [Ref 28:p. 328] 
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In recent versions of SMART the structuring of steps 1-4 have been emphasized. 
Recognizing the hierarchical nature of structures of objects and attributes frequently 
leads to versions of SMART that make use of value trees and hierarchical weighting 
procedures. [Ref. 20:p. 279] 
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APPENDIX (C): CRITERIUM DECISIONPLUS 


TM 


A. CAPABILITIES 


DecisionPlus^ implements two primary decision making methodologies, the 
Analytical Hierarchy Process (AHP) and a Multi-Attribute Utility Theory as implemented 
in the Simple Multi-Attribute Rating Technique (SMART). In this software package the 
primary differences between AHP and SMART lies in the different rating techniques used. 

When using SMART for decision making the problem is broken down into 
attributes, and single attribute evaluations are constructed by means of value 
measurements . A value tree structure is created to assist in defining the problem. The 
values are determined for each attribute and the software does aggregation of the model to 
provide results of the compared alternatives. [Ref 29:p. 33] The value tree starts with a 
goal and then branches out into criteria relating to that goal, and finally ending in 
alternatives for that goal. DecisionPlus™ is limited to seven levels including the goal level 
and the alternatives. The software will support a maximum of 255 blocks in the model 
and a maximum of 100 blocks on any level not including the alternative level. There can 
be a maximum of 50 alternatives and these also count against the total of255 blocks. 

[Ref 29:p 33] 

SMART provides a simplified method of employing MAUT techniques and allows 
the user to use a direct rating procedure for assessing single attribute values, and use 
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additive aggregation in calculating the preferred alternative. DecisionPlus^ also supports 
nonlinear functions in assigning values to the attributes. [Ref 29;p. 33] 

1. Brainstorming 

The first step in the decision process is to define the problem. DecisionPlus^s 
brainstorming capability assists the user in identifying the issues. The brainstorming 
session starts with a blank canvas ^ / des the user into defining a goal, important 

criteria, and alternatives. The goal and the criteria are grouped and connected by the user 
based upon the users perception of the relationships. Figure C-i is an example of a 
completed brainstorm session. [Ref 29;p. 44] 

2. Build the Hierarchy 

After using the brainstorming function the saved session automatically generates 
the hierarchy or structure. If the brainstorming fiinction was not used the structure can be 
created and edited through a user fiiendly mouse driven inteifiice. Figure 5-1 is an 
example of a completed hierarchy created by DecisionPlus™. [Ref 29:p. 44] 

3. Weight the Criteria 

Once the hierarchy is constructed the individual criteria must be assigned 
weights. The assignment of weights is a separate task but is done in DecisionPlus^'s 
Hierarchy session. By double clicking on a criteria or selecting rate sub-criteria from the 
main menu, the Criterion Rating window appears. In this window the subject is presented 
with a customizable view to elicited the rating information. Figure C-2 is an example of 
the Criterium Rating Window. 
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Figure C-1 Bminstorm-Gniph 

a. The Rating Views 

DecisionPlus^ provides the c^)ability to select between three different 
rating views. These views are selectable and are not mutually exclusive. 

(1) Numerical View 

In the numerical view the criterion that are being rated appear next to 
a box where a numerical weighting value can be entered. The numerical range of the 
box is selectable and unless modified it defaults to a 0.00 to 100.00 scale. 
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Figure C>2 Criteriou Rating Window 

(2) Graphical View 


In the graphical view the subject is presented with the sub-criterion 
next to a sliding bar. The evaluation is done by using a mouse to move the position of 
the bar to indicate the rating. 

(3) Verbal View 


In the verbal view six different verbal measurements can be assigned, 
each with its own numerical scale. The subject is presented with the sub-criteria next 
to a verbal measure in a pull down menu box. Opening the menu bar reveals the other 
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veibal measurements available for that sub-criterion with the currently selected one 
highlighted. Figure C-2 is an example of the presentation with the verbal view with the 
optional descr4>tive sentence. 

(4) Descriptive Sentence 

The Descriptive sentence is a sentence describing the rating logic as 
it relates to your goal. It uses the wording of the verbal scale selected to describe how 
one sub-criterrion is to be rated against another sub-criterion. Upon selecting a 
different verbal scale, or changing the ratings, the wording in the descriptive sentence 
changes also. [Ref. 29:p. 128] 

4. Review the Results 

After the hierarchy has been rated the results can be reviewed in one of 
several different forms. The results can be viewed as discrete values representing the 
preferences of the alternatives, or a view of the contributions screen. The contribution 
screen shows the contribution to each alternative preference based on the criteria at a 
given level in the hierarchy. [Ref. 29:p. 47] 

5. Sensitivity Analysis 

DecisionPlus™ supports checking for reasonableness of the decision with its 
Sensitivity Analysis function. The sensitivity analysis determines how sensitive the 
decision is to changes in the values assigned to the criteria. Upon selecting Sensitivity 
Analysis, DecisionPlus™ shows a list of the criteria with a metric that measures the 
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sensitivity of tte result when a change to the value of the child criteria is made. The 
list is prioritized in ortter of most critical to least critical to focus attention on the 
criteria that can influence the decision the most. fUef. 29:p. 48] 

6. Document the Decision 

DecisionPlus™ provides a complete report generation program to display the 
results of rating or the generation of the hierarchy chart. Some of the printable graphs 

and rqx)rts are: 

• Hierarchy - Graph 

• Hierarchy - Data 

• Hierarchy > Notes & Rules 

• Hierarchy - Results Graph 
■ Hierarchy • Results Data 

• Hierarchy - Sensitivity Graph 

• Hierarchy • Uncertainty Inputs 

• Hierarchy - Uncertainty Results 

• Hierarchy - Uncertainty Data 

• Hierarchy - Level Contributions 

• Hierarchy - Uncertainty Sensitivity 

By selecting the rqx)it option instead of the single itmns listed above a 
combination of any of the above can be combined into a r^it. [Ref. 29:p. 21] 
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APPENDIX (D): DATA 


Appendix D provides the data generated in the initial, intermediate and final steps of the 
calculations. This section displays the responses fi'om the subjects and other statistical data. 
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